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Abstract 

The background activity of a cortical neural network is modeled by 
a homogeneous integrate-and-fire network with unreliable inhibitory 
synapses. Numerical and analytical calculations show that the net- 
work relaxes into a stationary state of high attention. The majority 
of the neurons has a membrane potential just below the threshold; 
as a consequence the network can react immediately - on the time 
scale of synaptic transmission- on external pulses. The neurons fire 
with a low rate and with a broad distribution of interspike intervals. 
Firing events of the total network are correlated over short time pe- 
riods. The firing rate increases linearly with external stimuli. In the 
limit of infinitely large networks, the synaptic noise decreases to zero. 
Nevertheless, the distribution of interspike intervals remains broad. 

1 Introduction 

A neural network processes information by cooperating neurons, which 
interact by exchanging spikes of action potentials. Numerous experi- 
mental and theoretical findings suggest that a cortical neural network 
has a background activity of neurons firing irregularly at low rates 
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[HE]. On top of this noisy activity, information is processed. It is still 
not known how a neural network codes, stores and processes informa- 
tion. For instance, information in the cortex may be coded in patterns 
of neuronal firing rates (Hebbian cell assemblies) or in spatiotemporal 
patterns of spikes (synfire chains). But, evidently, all this occurs in 
a state of noisy activity. Therefore it is important to understand the 
properties of the background activity. 

Quantitative estimates of the rate and size of the synaptic events in 
the cortex suggest that the irregular background activity stems from a 
balance between excitatory and inhibitory postsynaptic currents |X3|, 
Hlj . Both of these currents are large, but they compensate and the 
activity of the network is triggered by the fluctuations of the total 
current. Recent model calculations showed how to integrate Hebbian 
cell assemblies or synfire chains in a balanced network [3j d] . 

However, experimental evidence for balanced synaptic input is still 
not available and it is not understood whether and how a neural net- 
work is able to compensate large synaptic currents of different direc- 
tions [131 [T7] . In addition, inhibitory pulses can shunt excitation very 
effectively. Therefore I follow a different approach and investigate a 
network with inhibitory synapses, only. At a first step, the investiga- 
tions concentrate on one single mechanism: unreliable inhibition. 

It is text book knowledge that synaptic transmission is a stochastic 
process. In fact, experiments on single synapses indicate that synapses 
transmit siganls with a probability which can be as low as a few per- 
cent [H [2]. Only if an incoming spike is repeated in a short time 
interval, the transmission probability increases [12J. This indicates 
that the background activity is driven by unreliable synapses whereas 
for information processing teh network improves synaptic reliability. 

In this paper I investigate how unreliable inhibition effects the 
collective properties of the network, hence I do not consider any kind 
of additional spatial or temporal noise, neither disorder in the synaptic 
connections and thresholds nor noise in the external stimulus. 

One of the simplest models to describe the cooperative properties 
of interacting neurons is a network of integrate-and-fire (IF) units [8]. 
A single neuron is driven by an excitatory input and generates and 
transmits spikes. The total network may have unexpected complex 
properties which are not obvious from a single model neuron. 

From a more general point of view, IF networks are networks of 
pulse coupled oscillators, and a large amount of work has been de- 
voted to the investigation of the dynamics of nonlinear oscillators [15]. 
Several interesting phenomena have been found for IF-networks, for 
instance synchronization [10] . phase locking, clustering [7], fast global 
oscillations [6], stochastic resonance, deterministic and transient chaos 
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PH Q2] and 

In this paper I investigate the stationary state of a network of 
inhibitory neurons. The effect of unreliable inhibitory synapses on 
the global properties of the network is studied numerically as well 
as analytically A single synapse consists of stochastic components; 
it transmits signals with a probability which can be as low as a few 
percent [HEIE]- Any other microscopic mechanisms are omitted in 
the model, like synaptic delay, distribution of thresholds and synaptic 
strengths. The effect of excitatory input is modeled by a constant 
stimulus. 

Neither the model nor the analytic methods of this paper are new. 
Nevertheless, a new phenomenon is observed which, to my knowledge, 
was not discussed before: A network with inhibitory couplings relaxes 
to a stationary state of high attention. After a few spikes per neuron, 
the distribution of membrane potentials is sharply peaked just below 
threshold. As a consequence, the system is able to react immediately, 
i.e. on the time scale of spike generation, to tiny external pulses. 

In the model of this paper the synaptic noise disappears in the 
limit of infinitely large networks. However, the system adjusts to a 
stationary state where the distribution of interspike intervals remains 
broad. 



2 The model 

In our model, each neuron is described by the following differential 
equation for the membrane potential V(t) [5], 

r f" = ~ v{t) + v °° (1) 

t is the relaxation time for the membrane potential and the driving 
potential models a constant excitatory stimulus. When the mem- 
brane potential V(t) reaches the threshold value 6 the neuron fires a 
spike and resets its potential to the value V r . 

A homogenous network is studied where each neuron has the pa- 
rameters t = 10 ms, V r = —70 mV, = —50 mV and 9 = —51 mV. 
Hence, without any interactions, each neuron would fire periodically 
with the time period T = rln20 ~ 30 ms which corresponds to a 
frequency of 33 Hz. 

I consider a homogeneous network of N mutually coupled neurons. 
If any neuron fires, it sends its spike to its synapses which are con- 
nected to all other neurons. The synapse is unreliable, it transmits the 
signal with a probability p, only. If a signal is transmitted, it reduces 
the postsynaptic potential by an amount J. In our simulations, I use 
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the parameters N = 10000, p = 0.5 and J = 0.002 mV. Hence, each 
synapse is extremely weak; but the effective strength pJN = lOmV is 
of the order of the difference between threshold and reset potentials. 

Between two consecutive pulses in the total the network, each neu- 
ron follows Eq. ([T]) which is easily solved. The time needed to increase 
the potential V(t\) to a value Vfe) is given by 



This equation allows a simple numerical simulation of the total net- 
work without solving any differential equation. At each computational 
step, the algorithm searches for the neuron with the maximal values 
of V(t), calculates the time this neuron needs to reach the threshold 
9 and resets its potential to the value V r . Then the potentials of all 
other neurons are calculated from Eq.([2]) and, with probability p, their 
values are reduced by the amount of J. This process is iterated until 
the system has reached a stationary state. Then the distributions of 
potentials and interspike intervals are recorded. 



The homogeneous network of IF-neurons with the parameters given 
before has been simulated numerically. I start from a flat distribution 
of potentials between reset and threshold value. After a few spikes 
per neuron the network has lost the memory to its initial state and 
has relaxed to a stationary state. Note that our model is stochastic, 
hence I have to discuss distributions of quantities of interest. 

In the following I show results for the distribution of membrane 
potentials and interspike intervals, both of which are accessible for real 
neurons. Fig[T] shows the membrane potential of one single neuron as 
a function of time. Since the network is homogeneous all neurons have 
identical properties. Between two spikes, each neuron receives about 
5000 pulses from the 9999 other neurons of the network. Each pulse 
reduces its potential by the amount of J = 0.002 mV. 

I see that a typical neuron increases its potential exponentially 
fast to a kind of plateau immediately below threshold. The stochastic 
input leads to fluctuations of the membrane potential. Whenever this 
potential exceeds the threshold value the neuron fires. Note that the 
average spike interval increases from 30 ms without couplings to about 
100 ms. Obviously, the inhibitory pulses reduce the firing rate. But 
less obvious, most of the neurons have a membrane potential close 
to the threshold value. This is shown in Fig. The distribution of 





3 Results of the simulations 
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Figure 1: Membrane Potential V(t) as a function of time for a network of 
10000 inhibitory neurons with unreliable synapses. As soon as the potential 
reaches the threshold value 9 = — 51mV the neuron emits a spike and resets 
its potential to the value V r = — 70mV. Between two spikes,the potential is 
reduced about 5000 times by the value of J = 0.002mV 

membrane potentials has a peak which is only 0.05 mV below thresh- 
old. 

Note that the initial distribution was flat between -70 mV and -50 
mV. After a short time the system has relaxed to a stationary where 
most of the neurons have a potential immediately below threshold. As 
a consequence, a large fraction of neurons can react immediately to 
tiny pulses of the order of a fraction of millivolts. In Fig [3] the fraction 
of neurons is shown which fires immediately after a excitatory pulse of 
a given strength J ex . For comparison, I have included the correspond- 
ing curve for the initial state. Fig. U] shows the corresponding spike 
pattern for J ex = 0.5mV. All of the neurons which have responded to 
the pulse excitation remain quiet for at least 50 ms, see Fig [5) These 
results show that the network has relaxed to a state of high attention 
and can react to incoming small pulses on the time scale of synaptic 
transmission. 

The distribution of interspike intervals T is shown in Fig. [5j There 
are no intervals below 50 ms and the distribution decays exponentially 
fast for large values of T. Its mean value is < T >= 97.6 ms and its 
standard deviation a = 29.0ms, which gives a CV-value of 0.3. Since 
the network has identical neurons, the average time s between two 
consecutive spikes in the total network is < s >=< T > fN — 0.01 
ms. Usually one assumes that the spikes arriving at a single neuron 
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Figure 2: Histogram of the membrane potentials of the network at some 
specified time. Initially, the distribution was flat between -70mV and -51 
mV. In the stationary state, shown here, most of the neurons have a po- 
tential immediately below threshold threshold. The inset shows the same 
distribution close to threshold. 



are uncorrelated, which would yield a Poisson, i.e. an exponential 
distribution of the intervals s between synaptic events. Fig. [6] shows 
that this is not true. Although small values of s are more frequent than 
larger ones, the distribution is not Poissonian indicating correlations 
between events. In fact, the inset of Fig. [6] shows the correlations 
between consecutive spikes. The spike intervals of the total network 
are ordered, (si, s%, S3, ...Sj, ..), and Fig. [6] shows the cross correlation 



C r k \ _ < s j+kSj > - < Sj > 
< S? > - < Sj > 2 



(3) 



Correlations can also be observed from the statistics of spike counts. 
For instance, the Fano factor F is defined as the ratio of the variance 
divided by the average number of spikes observed in some time inter- 
val AT. For a Poisson process one has F = 1 independent of AT. I 
have recorded the number of network events for AT = 1ms and find 
a much smaller Fano factor of F = 0.15. Hence the distribution of 
the number of spikes is much smaller than the corresponding one for 
uncorrelated synaptic inputs. 

In the context of pulse coupled oscillators the phase <p of an oscil- 
lator is of interest. When a reference neuron fires at time t the phase 
is defined as <pk = t — tk where neuron k has fired at at previous time 
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Figure 3: The fraction of neurons which fires immediately after an excitatory 
pulse of a given value J ex . The network with stochastic synapses (lower val- 
ues) is compared with a corresponding deterministic network (upper values). 
The dotted line shows the corresponding fraction of neurons for the initial 
state with a flat distribution of potentials. 

ifc. The distribution of phases is shown in Fig. [71 The distribution 
of a deterministic network which is flat between and 100 ms (see 
below) is rounded due to synaptic noise. 

It may be interesting to compare the properties of our stochastic 
model with the corresponding deterministic one, i.e. with the model 
with parameters p = 1 and J = 0.5. The effective strength pJN = 10 
mV is identical but the synapses function completely reliable. Our 
simulations show that the deterministic model relaxes to a periodic 
state with a flat distribution of phases. Each neuron fires periodically 
with the period T = 100.04 ms. The time s between events of the 
total network is constant, s = T/N = 0.10004 ms. Hence, both of the 
distributions of interspike as well as interevent intervals have a sharp 
peak at a single value. This property of the deterministic network is 
definitely at variance with the behavior of real neural networks. 

Nevertheless, the distribution of membrane potentials is similar to 
the one of the corresponding stochastic network, Fig. El Only very 
close to threshold one observes deviations. Note that a fiat distri- 
bution of phases transforms to a distribution of potentials which is 
proportional to l/(fi — u) with a cutoff above the value 6. The param- 
eter \x is slightly larger than the value and will be calculated in the 
following section. 
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Figure 4: Spike patterns of one hundred neurons (out of 10000). At time 
t = 400ms, an excitatory pulse is applied which increases all postsynaptic 
potentials by the amount of 0.5 mV. According to the previous figure El 
about 60% of the neurons fire immediately. 



Although our model is rather simple it cannot be calculated analyt- 
ically. Even two neurons with unreliable synapses yield a complex 
multifractal distribution of interspike intervals which can only be cal- 
culated numerically [11] . But analytic approximations are possible 
and help to understand the influence of the model parameters on the 
behavior of the network. 

It is well known that a stochastic network may be approximated 
by a diffusion process of the membrane potential |181 [5] . Assum- 
ing uncorrelated synaptic input one can describe the time dependent 
distribution of potentials by a Fokker-Planck equation for an Ornstein- 
Uhlenbeck process. Spike intervals correspond to first passage times. 
Although, even for this approximation, there is no closed expression 
for the distribution of spike intervals, one can calculate its mean value 
from the solution of an ordinary differential equation. 

Two parameters enter this approximation: The mean value and the 
variance of the synaptic input. Let us first ignore the fluctuations and 
consider the membrane potential, Eq.([T]), averaged over the stochastic 



4 Analytic approximations 
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Figure 5: Histogram of the intervals between spikes of a single neuron. 



synaptic input. Each spike in the network adds a pulse pj to the 
potential of every neuron. The average time between two pulses is 
given by < s >=< T > /N, hence I obtain the following equation 



rpJN 
<T> 



(4) 



The mean value of the membrane potential moves exponentially fast 
to the value 

V Tj9N fK\ 

until it crosses the threshold. If I would neglect fluctuations, the spike 
interval would follow from Eq.(j2J), which yields 



< T > 



In 



V' 



rpJN 
<T> 



Vr 



rpJN 
<T> 



(6) 



This self-consistent equation for the interspike interval is exact for the 
deterministic model in the limit of a large network, N — > oo with 
JN =const. Note that in this case, the system relaxes to a periodic 
state with only one single spike interval T. If the effective synaptic 
strength JN is large, i.e. if the spike interval T is much large than r, 
the denominator of Eq.flBJ) approaches zero and one obtains 



< T > 



pJN 



(7) 



For a given value of J, the spike interval is proportional to the size 
of the network and its inverse, the spike rate, is proportional to the 
difference between stimulus and threshold potential. 
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Figure 6: Histogram of the intervals between spikes of the total network. 
The inset shows the corresponding cross correlation of consecutive spikes. 

Our numerical simulations showed that the deterministic network 
relaxes to a state with a flat distribution of phases (times after firing) . 
Using this fact, one immediately derives the distribution of membrane 
potentials from Eq.flT]) with the result 

tN 1 

P(V) = — -—y for V r <V<9 (8) 

In the limit of infinitely large networks the potential \i approaches the 
threshold, hence in this limit almost all neurons have a potential very 
close to threshold. In fact, Eqs. (|6)7p give the fraction P(e) of neurons 
which have a membrane potential in the interval [6 — e, 9], 

hu> - g + e) - hu> - 0) 
{ ) InQu - V r ) - ln(/i - 6) { ) 

This fraction approaches P(s) = 1 with /i — > 0. 

For the stochastic version of the model, the simulations of the pre- 
vious section yielded a broad distribution of spike intervals T which is 
caused by the fluctuations of synaptic inputs. Therefore, in the ana- 
lytic approximation, fluctuations of the synaptic input are included by 
adding noise to Eq.([T]) with mean value fi and variance a 2 . The value 
of the mean [i is given by Eq.(|5]) and the variance can be estimated 
by assuming a Poisson statistics, one finds [3] 

rJgN 2 t2 pNr 
M = — — , a = J — — — (10) 

P °°<T>' <T> v; 
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Figure 7: Histogram of the phases of the neurons. 



The mean spike interval is obtained from the self-consistent equation 



< T > 



due~ u 



^2y e u _ 2y r u 



(11) 



with 



fje 



a 



JJr 



(12) 



If ye is large, i.e. if the fluctuations of the potential are small compared 
to the difference between the driving potential \x and the threshold 0, 
then the mean first passage time can be approximated by 



< T > 



(13) 



For the parameters of our simulations of the previous section, Eqs. 
(|ll|13p give < T >= 94.6 ms and < T >= 93.5 ms which is in rea- 
sonable agreement with the mean spike interval of the simulations, 
< T >= 97.6 ms. The effective average stimulus /x is only 0.05 mV 
below the threshold potential 6, but this difference is still large com- 
pared to the fluctuations, one finds yg = 1.23. 

Note that, contrary to the deterministic case, the mean potential 
relaxes to a value /U below the threshold 8. The spikes are generated 
by fluctuations which cross the threshold. Nevertheless, in both cases, 
the value of fi is close to 9 which almost gives an identical average 
spike interval, Eq.©. In particular, the spike rate increases linearly 
with the stimulus V^. 
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It might be interesting to consider the limit of large networks, 
N — > oo with constant JN = C. Inserting the average spike interval, 
Eq.©, into Eq.flT3]) yields 



Hence the fluctuations of the synaptic noise show an unusual be- 
haviour in this limit: Its variance decreases with increasing system 
size N. However, the fluctuations relative to the difference between 
driving potential fi and threshold 6 remain constant, yielding a con- 
stant mean spike interval < T >. The fluctuations do not disappear 
in the thermodynamic limit N — > oo. 



A homogeneous neural network with purely inhibitory synapses re- 
laxes to a state of high attention. A large fraction of neurons have a 
membrane potential which is just below threshold. Hence the network 
can immediately - on the time scale of synaptic transmission - react 
on external excitatory pulses. 

Although, at any time, the majority of the neurons accumulate 
closely below threshold, each single neuron fires irregularly with a low 
rate. In our model the broad distribution of spike intervals stems 
from synaptic unreliability; a synapse transmits the incoming spike 
with some probability, only. The numerical simulations show that the 
spiking events of the total network are correlated in time. Uncorre- 
lated synaptic noise still leads to correlated spike intervals, even for 
infinite-range models. 

For our model, in the thermodynamic limit any neuron receives 
an infinite number of synaptic inputs. Usually, one argues that, in 
this limit, the fluctuations of the synaptic input can be ignored. Our 
model, however, shows that even for an infinite number of inhibitory 
inputs the fluctuations of the membrane potential have a large effect. 
The network adjusts itself such that there remains a broad distribution 
of spike intervals in this limit. 

The stationary state of the network is a pool of neurons firing 
irregularly with a low rate. The firing rate increases linearly with 
a global external stimulus. The network can immediately react to 




(14) 



(15) 



(16) 
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external pulses. Our model contains unreliable inhibitory synapses, 
only. Hence a network does not need to balance large excitatory and 
inhibitory synaptic currents to achieve these properties. 

Our investigations concentrated on a very simple model: Integrate- 
and-fire neurons with unreliable inhibitory synapses which couple all 
neurons. Each neuron has identical parameters. One may wonder 
whether and how the properties of the network change when the model 
is extended to other mechanisms. In fact, much is known about IF- 
networks. For instance, synaptic delay leads to clusters of synchro- 
nized activity [7] and to oscillating firing rates [6J. A distribution 
of the number of short-range synaptic contacts increases the correla- 
tions between consecutive spike intervals |9J. Excitatory synapses can 
produce synfire chains, i.e. waves of synchronized activities moving 
through the network [TJ [3]. When the excitation is too strong, the 
neurons synchronously fire at a high rate [5] . A distribution of thresh- 
old values obviously leads to a distribution of firing rates. In fact, I 
find that in this case, the stationary state of the network contains a 
fraction of neurons which never fire. 

All these extensions modify the properties of state of irregularly 
firing neurons. However, one of the main questions remains unsolved: 
How does the network process information on top of this background 
activity? 
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